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D 1 Abstract 

C/D ■ 

I explore the possibility that the laws of physics might be laws of 
inference rather than laws of nature. What sort of dynamics can one derive 
from well-established rules of inference? Specifically, I ask: Given relevant 
information codified in the initial and the final states, what trajectory is 
the system expected to follow? The answer follows from a principle of 
inference, the principle of maximum entropy, and not from a principle of 
\^ ' physics. The entropic dynamics derived this way exhibits some remarkable 

, formal similarities with other generally covariant theories such as general 

Q\ • relativity. 
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1 Introduction 

q-i The study of changes in the natural world, dynamics, is divided among several 

distinct disciplines. Thermodynamics, for example, considers changes between 
bJT), special states, the so-called states of equilibrium, and addresses the question of 

which final states can be reached from any given initial state. Mechanics studies 
the changes we call motion, chemistry deals with chemical reactions, quantum 
^ ' mechanics with transitions between quantum states, and the list goes on. 

In all of these examples we want to predict or explain the observed changes 
on the basis of information that is codified in a variety of ways into what we 
call the states. In some cases the final state can be predicted with certainty, in 
others the information available is incomplete and we can, at best, only assign 
probabilities. 

The theory of thermodynamics holds a very special place among all these 
forms of dynamics. With the development of statistical mechanics by Maxwell, 
Boltzmann, Gibbs and others, and eventually culminating in the work of Jaynes 
thermodynamics became the first clear example of a fundamental physical 
theory that could be derived from general principles of probable inference. The 
entire theory follows from a clear idea of the subject matter, that is, an appro- 
priate choice of which states one is talking about, plus well-known principles of 
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inference [Q, namely, consistency, objectivity, universality and honesty. These 
principles are sufficiently constraining that they lead to a unique set of rules 
for processing information: these are the rules of probability theory jj| and the 
method of maximum entropy Q Q] . 

There are strong indications that a second example of a dynamics that can 
be deduced from principles of inference is afforded by quantum mechanics 
Many features of the theory, traditionally considered as postulates, follow from 
the correct identification of the subject matter plus general principles of infer- 
ence. Briefly, the goal of quantum mechanics is not to predict the behavior of 
microscopic particles, but rather to predict the outcomes of experiments per- 
formed with certain idealized setups. Thus, the subject of quantum theory is 
not just the particles, but rather the experimental setups. The variables that 
encode the information relevant for prediction are the amplitudes or wave func- 
tions assigned to the setups. These ingredients plus a requirement of consistency 
(namely, that if there are two ways to compute an amplitude, the two results 
should agree) supplemented by entropic arguments are sufficient to derive most 
of the standard formalism including Hilbert spaces, a time evolution that is 
linear and unitary, and the Born probability rule. 

If quantum mechanics, deemed by many to be the fundamental theory, can 
be derived in this way, then it is possible, perhaps even likely, that other forms of 
dynamics might ultimately reflect laws of inference rather than laws of nature. 
Should this turn out to be the case, then the fundamental equations of change, 
or motion, or evolution as the case might be, would follow from probabilistic and 
entropic arguments and the discovery of new dynamical laws would be reduced 
to the discovery of what is the necessary information for carrying out correct 
inferences. Unfortunately, this search for the right variables has always been 
and remains to this day the major stumbling block in the understanding of new 
phenomena. 

The purpose of this paper is to explore this possible connection between the 
fundamental laws of physics and the theory of probable inference: Can dynamics 
be derived from inference? Rather than starting with a known dynamical theory 
and attempting to derive it, I proceed in the opposite direction and ask: What 
sort of dynamics can one derive from well-established rules of inference? 

In section 2 I establish the notation, define the space of states, and briefly 
review how the introduction of a natural quantitative measure of the change 
involved in going from one state to another turns the space of states into a metric 
space ||. (Such metric structures have been found useful in statistical inference, 
where the subject is known as Information Geometry 0, and in physics, to study 
both equilibrium and nonequilibrium thermodynamics |9j.) 

Typically, once the kinematics appropriate to a certain motion has been 
selected, one proceeds to define the dynamics by additional postulates. This is 
precisely the option I want to avoid: in the dynamics developed here there are 
no such postulates. The equations of motion follow from an assumption about 
what information is relevant and sufficient to predict the motion. 

In a previous paper Q I tackled a similar problem. There I answered the 
question: 
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Ql: Given the initial state and that the system evolves to other states, what 
trajectory is the system expected to follow? 

This question implicitly assumes that there is a trajectory and that information 
about the initial state is sufficient to determine it. The dynamical law follows 
from the application of a principle of inference, the method of maximum entropy 
(ME), to the only information available, the initial state and the recognition that 
motion occurred. Nothing else. The resulting 'entropic' dynamics is very simple: 
the system moves continuously and irreversibly along the entropy gradient. 

Thus, the honest, correct answer to the inference problem posed by question 
Ql has been given, but the equally important question 'Will the system in fact 
follow the expected trajectory?' remained unanswered. Whether the actual 
trajectory is the expected one depends on whether the information encoded 
in the initial state happened to be sufficient for prediction. Indeed, for many 
systems, including those for which the dynamics is reversible, more information 
is needed. 

In section 3 we answer the question: 

Q2: Given the initial and the final states, what trajectory is the system ex- 
pected to follow? 

Again, the question implicitly assumes that there is a trajectory, that in mov- 
ing from one state to another the system will pass through a continuous set of 
intermediate states. And again, the equation of motion is obtained from a prin- 
ciple of inference, the principle of maximum entropy, and not from a principle of 
physics. (For a brief account of the ME method in a form that is convenient for 
our current purpose see p0|.) The resulting 'entropic' dynamics also turns out 
to be simple: the system moves along a geodesic in the space of states. This is 
simple but not trivial: the geometry of the space of states is curved and possibly 
quite complicated. 

Important features of this entropic dynamics are explored in section 4. We 
show that there are some remarkable formal similarities with the theory of 
general relativity (GR). For example, just as in GR there is no reference to an 
external physical time. The only clock available is provided by the system itself. 
It turns out that there is a natural choice for an 'intrinsic' or 'proper' time. It is a 
derived, statistical time defined and measured by the change itself. Intrinsic time 
is quantified change. This entropic dynamics can be derived from a Jacobi-type 
principle of least action, which we explore both in Lagrangian and Hamiltonian 
form. Just as in GR there is invariance under arbitrary reparametrizations - a 
form of general covariance - and the entropic dynamics is an example of what 
is called a constrained dynamics. 

2 Quantifying change: geometry 

In this section we briefly review how to quantify the notion of change (for more 
details see 0). The idea is simple: since the larger the change involved in going 
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from one state to another, the easier it is to distinguish between them, we claim 
that change can be measured by distinguishability. Next, using the ME method 
one assigns a probability distribution to each state. This transforms the problem 
of distinguishing between two states into the problem of distinguishing between 
the corresponding probability distributions. The solution is well-known: the 
extent to which one distribution can be distinguished from another is given by 
the distance between them as measured by the Fisher-Rao information metric 
[[n] JfJI [[7) . Thus, change is measured by distinguishability which is measured 
by distance. 

Let the microstates of a physical system be labelled by x, and let m{x)dx 
be the number of microstates in the range dx. We assume that a state of the 
system (i.e., a macrostate) is defined by the expected values A a of some tia 
appropriately chosen variables a a (x) (a = 1, 2, ... , tia), 

(a a ) = ( dxp{x)a a {x) = A a . (1) 



A crucial assumption is that the selected variables codify all the information rel- 
evant to answering the particular questions in which we happen to be interested. 
This is a point that we have made before but must be emphasized again: there is 
no systematic procedure to choose the right variables. At present the selection 
of relevant variables is made on the basis of intuition guided by experiment; it 
is essentially a matter of trial and error. The variables should include those 
that can be controlled or observed experimentally but there are cases where 
others must also be included. The success of equilibrium thermodynamics, for 
example, derives from the fact that a few variables are sufficient to describe 
a static situation, and being few, these variables are easy to identify. In fluid 
dynamics, on the other hand, the selection is more dificult. One must include 
many more variables, such as the local densities of particles, momentum, and 
energy, that are neither controlled nor usually observed. 

The states form an nA-dimensional manifold with coordinates given by the 
numerical values A a . To each state we can associate a probability distribution 
p{x\A). The distribution that best reflects the prior information contained in 
m(x) updated by the information A a is obtained by maximizing the entropy 

S\p:m} = - f dx P (x) log -^p-. (2) 

J 11 1 { X j 

subject to the constraints ([!]). The result is 

p(x\A) = ±m(x)e- x <* aa W, (3) 
where the partition function Z and the Lagrange multipliers A Q are given by 
Z(X) = ( dxm{x)e- x « a °' {x) and - dl °^ Z = A a . (4) 

J OA a 

Next, we argue that the change involved in going from state A to the state 
A + dA can be measured by the extent to which the two distributions can be 
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distinguished. As discussed in pj, except for an overall multiplicative constant, 
the measure of distinguishability we seek is given by the 'distance' d£ between 
p(x\A) and p(x\A + dA), 

d£ 2 = g af} dA a dA 13 , (5) 

where 

[ ,1 i a\ d\ogp{x\A) d\ogp(x\A) 
g al3 = J dxp(x\A) — — p (6) 

is the Fisher-Rao metric jllj][jl^. It turns out that this metric is unique: it is 
the only Ricmannian metric that adequately reflects the fact that the states A 
are not 'structureless points', but happen to be probability distributions. 

To summarize: the very act of assigning a probability distribution p(x|A) to 
each state A, automatically provides the space of states with a metric structure. 



3 Dynamics and intrinsic time 

Given the initial and the final states, what trajectory is the system expected to 
follow? The key to answering this question lies in the implicit assumption that 
there exists a trajectory or, in other words, that large changes are the result of 
a continuous succession of very many small changes. Thus, the difficult problem 
of studying large changes is reduced to the much simpler problem of studying 
small changes. 

Let us therefore focus on small changes and assume that the change in going 
from the initial state At to the final state Af = Ai + AA is small enough that 
the distance A£ between them is given by 

At 2 = g al) AA a AA^ . (7) 

To find which states are expected to lie on the trajectory between Ai and Af 
we reason as follows. In going from the initial to the final state the system must 
pass through a halfway point, that is, a state A that is equidistant from Ai 
and Af (see fig. la). The question is which halfway state should we choose? An 
answer to this question would clearly determine the trajectory: first find the 
halfway point, and use it to determine 'quarter of the way' points, and so on. 

Next we notice that there is nothing special about halfway states. We could 
equally well have argued that in going from the initial to the final state the 
system must first traverse a third of the way, that is, it must pass through a 
state that is twice as distant from Af as it is from Ai . In general, we can assert 
that the system must pass through intermediate states A^ such that, having 
already moved a distance di away from the initial Ai, there remains a distance 
L)d£ to be covered to reach the final Af. Halfway states have u> = 1, 'third of 
the way' states have ui = 2, and so on (see fig. lb). 

It appears that each different value of u> provides a different criterion to select 
the trajectory. If there is a trajectory and there are several ways to determine 
it, consistency demands that all these ways should agree: in the end we must 
verify that the selected trajectory is independent of lu or else we have a problem. 

Our basic dynamical question Q2 can be rephrased as follows: 
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Figure 1: The geometry of dynamics: in going from the initial At to the final 
state A f the system must pass through states A (dashed line) that are equidis- 
tant between them (a), and it must also pass through states A^ (dashed circle) 
that are u) times as far from Af as they are from Ai (b). 

Q2' The system is initially in state p{x\Ai) and we are given the new informa- 
tion that the system has moved to one of the neighboring states in the 
family p{x\A ul ). Which p(x\A ul ) do we select? 

Phrased in this way it is clear that this is precisely the kind of problem to be 
tackled using the ME method. Recall The ME method is a method for 
processing information and change our minds. It allows us to go from an old 
set of beliefs, described by the prior probability distribution, to a new set of 
beliefs, described by the posterior distribution, when the information available 
is just a specification of the family of distributions from which the posterior 
must be selected Jl3j . In the more traditional applications of the method this 
family of posteriors is constrained or defined by the known expected values of 
some relevant variables, but this is not necessary, the constraints need not be 
linear functionals. Here the constraints are defined geometrically. 

An important question that should arise whenever one contemplates using 
the ME method is which entropy should one maximize. Since we want to select 
a distribution p{x\ A) the entropies to be considered must be of the form 
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This is the entropy oi p(x\A) relative to the prior q(x). The interpretation of 
q(x) as the prior follows from the logic behind the ME method itself. Recall 
[ p"o[ : In the absence of new information there is no reason to change one's 
mind. When there are no constraints the selected posterior distribution should 
coincide with the prior distribution. Since the distribution p that maximizes 
S[p : q] subject to no constraints is p oc q, we must set q(x) equal to the prior. 

Coming back to our dynamical problem, suppose we know that the system 
is initially in state p(x\Ai) and we are not given the information that the sys- 
tem moved, then we have no reason to suspect that any change has occurred. 
Therefore the prior q(x) should be chosen so that the maximization of S[p : q] 
subject to no constraints yields the posterior p — p(x\Ai). The correct choice is 
q{x) = p(x\Ai). 

Now we are ready to tackle the question Q2': the answer is obtained by 
maximizing the entropy 

S[A : Aj] = - J dxp(x\A) log (9) 

subject to the constraint A = A u . This presents no problems. It is convenient 
to write A u = A^ + dA and Aj — Ai + AA so that S[A U : Aj\ simplifies to 

S[Ai + dA : Ai] = ~g afi dA a dA\ (10) 

and the distances dli and dl t from A^ to A; and A f are given by 

di\ = g ap dA a dA 13 and di) = g af3 (AA a - dA a )(AA@ - dA 13 ). (11) 

Then, to maximize S[Ai + dA : Aj\ under variations of dA subject to the con- 
straint 



ujdti = dtf 
introduce a Lagrange multiplier A, 



(12) 

(-^£M dA a dA S) - A {uj 2 d^ - di})^ = . (13) 



We get 



dA a = yAA a where v = ; — . (14) 

A A l-w 2 + l/2A v ' 

The multiplier A, or equivalently the quantity \i is determined substituting back 
into the constraint (|l2|). From eq.(|ll) we get dli = an d d£f = (1 — x)^i 
and therefore jl4| 

X = and A = — ; . (15) 

Thus, the intermediate state A u selected by the maximum entropy method is 
such that 

dli + d£f = A£ . (16) 
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The geometrical interpretation is obvious: the triangle defined by the points 
Ai, Aw, and Af (fig.l) degenerates into a straight line. This is sufficient to 
determine a short segment of the trajectory: all intermediate states lie on the 
straight line between Ai and Af. The generalization beyond short trajectories 
is immediate: if any three nearby points along a curve lie on a straight line the 
curve is a geodesic. Note that this result is independent of the value us so the 
potential consistency problem we had identified earlier does not arise. 
To summarize, the answer to our question Q2 is simple and elegant: 

ED The expected trajectory is the geodesic that passes through the given initial 
and final states. 

This is the main result of this paper. As promised, in entropic dynamics the 
motion is predicted on the basis of a 'principle of inference', the principle of 
maximum entropy, and not from a 'principle of physics'. 

The dynamics ED was derived in an unusual way and one should expect 
some unusual features. Indeed, they become evident as soon as one asks any 
question involving time. For example, ED determines the vector tangent to the 
trajectory dA a /dt, but not the actual 'velocity' dA a /dt. The reason is not hard 
to find: nowhere in question Q2 nor in any implicit background information is 
there any reference to an external time t. 

Additional information is required if one is to find a relation between the 
distance I along the trajectory and the external time t. In conventional forms of 
dynamics this information is implicitly supplied by a 'principle of physics', by 
a Hamiltonian which fixes the evolution of a system relative to external clocks. 
But Q2 makes no mention of any external universe; the only clock available 
is the system itself, and our problem becomes one of deciding how this clock 
should be read. We could, for example, choose one of the variables A a , say A 1 , 
as our clock variable and arbitrarily call it intrinsic time. Ultimately it is best 
to define intrinsic time so that motion looks simple. 

A very natural definition consists in stipulating that the system moves with 
unit velocity, then the intrinsic time r is given by the distance £ itself, dr = di. 
Intrinsic time is quantified change. A peculiar consequence of this definition is 
that intervals between events along the trajectory are not a priori known, they 
are determined only after the equations of motion are solved and the actual 
trajectory is determined. This reminds us of the theory of General Relativity 
(GR). 

An important feature of GR is the absence of references to an external time. 
Given initial and final states, in this case the initial and final three-dimensional 
geometries of space, the proper time interval along any curve between them 
is only determined after solving the Einstein equations of motion Jl5| . The 
absence of an external time has been a serious impediment in understanding 
the classical theory - because it is not clear which variables represent the 
true gravitational degrees of freedom - and also in formulating a quantum theory 
[ p~7| - because of difficulties in defining equal-time commutators. 

In the following section we rewrite the entropic dynamics in Lagrangian and 
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Hamiltonian forms and we point out some further formal similarities between 
ED and GR. 



4 Formal developments 

The entropic dynamics can be derived from an 'action' principle. Since the 
trajectory is a geodesic, the 'action' is the length itself, 

J[A]= dr,L(A,A), (17) 

where 77 is an arbitrary parameter along the trajectory the Lagrangian is 

/ . . \ 1/2 . rlA a 

L{A,A) = (g a pA a AP) and A a = — . (18) 

The action J[A] is invariant under reparametrizations A(r/) ■—> A(f(rj)) pro- 
vided the end points are left unchanged, /(?7j) = rji and /(??/) = i)t . Indeed, 
when the transformation is infinitesimal, f{if) = r\ + s(rj), the corresponding 
change in the action, 



1/2 

5J = [g a0 A a A^ e(r)) 



if 

(19) 



vanishes provided £(%) = e(ri f) = 0. As pointed out in [ fL8[ there is an important 
distinction between the symmetries of a generally covariant theory such as GR 
and the internal symmetries of a proper gauge theory. The action of a generally 
covariant theory is invariant under those reparametrizations that are restricted 
to map the boundary onto itself; for proper internal gauge transformations 
there are no such restrictions. Thus ED shares with GR the fact that both are 
generally covariant theories. 

It is instructive to consider the analogous principle of least action for a 
nonrelativistic particle. The standard Hamilton's principle requires extremizing 
the action 

A^——-^)), (20) 

where t is 'physical' time, and the interval between initial and final states tf — t% 
is given. In contrast, Jacobi's principle of least action for a particle with energy 
E moving in a potential V(x) determines the trajectory by extremizing the 
action 

d v i^mS^^-J {E~V{x)f 2 . (21) 

There is no reference to any time t, the time interval between initial and fi- 
nal states is not given, and the parameter rj is unphysical and arbitrary. To 
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determine the temporal evolution along the trajectory requires an additional 
supplementary condition, 



m . dx a dx Tr/ , „ . . 

+V(aj)=£ - (22) 

Thus the ED action, eq.(|l7|), is an action of the Jacobi type. The natural choice 
for a supplementary condition that defines r and determines the evolution along 
the trajectory is 

dA a dA? , . 

a ^^r = 1 ■ ( 23 ) 

It is interesting that GR is also described by a Jacobi- type action JljJ. To 
explore this similarity further it is convenient to construct the canonical (i.e., 
Hamiltonian as opposed to Lagrangian) version of Jacobi's action. 
The canonical momenta are given by 



8L g af3 AP 



(24) 



r)A» / . • \l/2 

and have unit magnitude, 

g^airp = 1 . (25) 
The canonical Hamiltonian vanishes identically, 

iWA tt) = A a n a - L(A, A)=0, (26) 

because the Lagrangian is homogeneous of first degree in the A's. Physically 
this is not surprising: the generator of time evolution can be expected to vanish 
whenever there is no external time with respect to which the system could 
possibly evolve. We are led to consider the canonical action 

f S dt) (A a n a - H c Jj = f S dq A a ir a , (27) 

but eq.(|25|) implies that unconstrained variations of the momenta n a are not 
allowed. The correct variational principle requires to extremize the action 

f v t r • i 
I[A,tt,N}= dri A a ir a - N h(A, it) (28) 

where 

h(A,n) = ^n a n f3 -^ (29) 
and N(rj) are Lagrange multipliers that enforce the constraint 

h(A,w) = 0. (30) 
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for each value of rj. The overall factor of 1/2 in eq.(^) is introduced for later 
convenience; it amounts to rescaling TV. Variation of I[A,tt,N] with respect to 
A, 7r, and N yields the equations of motion, 

and eq.(|3^). Naturally there is no equation of motion for N and it must be 
determined from the constraint. We obtain, 

N=(g a pA a A^ 1 '\ (32) 

which, using the supplementary condition eq. (|2S|) , implies 

dr = Nd<q. (33) 

The analogue of N in GR is called the lapse function, it gives the increase of 
'intrinsic' time per unit increase of the unphysical parameter rj. In terms of r 
the equations of motion become 

dir a dh , dA a dh , . 

-r 1 = —^r and —r- = ^— ■ 34 

dr dA a dr dir a 

In reparametrization invariant or generally covariant theories there is no canon- 
ical Hamiltonian (it vanishes identically) but there are constraints. It is the 
constraints that play the role of generators of evolution, of change. Accord- 
ingly, the analogue of eq.(p3) in GR is called the Hamiltonian constraint. 



5 Final remarks 

I have provided an answer to the question 'Given the initial and final states, 
what is the trajectory followed by the system?' The answer follows from estab- 
lished principles of inference without invoking additional 'physical' postulates. 
The entropic dynamics thus derived turns out to be formally similar to other 
generally covariant theories: the dynamics is reversible; the trajectories are 
geodesies; the system supplies its own notion of an intrinsic time; the motion 
can be derived from a variational principle that turns out to be of the form of 
Jacobi's action principle rather than the more familiar principle of Hamilton; 
and the canonical Hamiltonian formulation is an example of a dynamics driven 
by constraints. 

To conclude one should point out that a reasonable physical theory must 
satisfy two key requirements: the first is that it must provide us with a set of 
mathematical models, the second is that the theory must identify real physical 
systems to which the models might possibly apply. The entropic dynamics we 
propose in this paper satisfies the first requirement, but so far it fails with respect 
to the second; it may be a reasonable theory but it is not yet 'physical'. There 
are formal similarities with the general theory of relativity and one wonders: Is 
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this a coincidence? Whether GR will in the end turn out to be an example of 
ED is at this point no more than a speculation. A more definite answer hinges 
on the still unsettled problem of identifying those variables that describe the 
true degrees of freedom of the gravitational field [pi |TjJ . 
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